10. Ethics in Experimentation

Ethics in Experimentation

Before you run an experiment, it's important to consider the ethical treatments
to which you subject your participants. Through the mid-20th century, exposure
of questionable and clearly unethical research in the social and medical
sciences spurred the creation of guidelines for ethical treatment of human
subjects in studies and experiments. While different fields have developed
different standards, they still have a number of major points in common:

• Minimize participant risk: Experimenters are obligated to construct
  experiments that minimize the risks to participants in the study. Risk of harm
  isn't just in the physical sense, but also the mental sense. If an experimental
  condition has potential to negatively affect a participant's emotions or
  mentality, then it's worth thinking about if the risks are really necessary to
  perform the desired investigation.

• Have clear benefits for risks taken: In some cases, risks may be
  unavoidable, and so they must be weighed against the benefits that may come
  from performing the study. When expectations for the study are not clearly
  defined, this throws into question the purpose of exposing subjects to risk.
  However, if the benefits are shown to be worth the risks, then it is still
  possible for the study to be run. This often comes up in medicine, where test
  treatments should show worthy potential against alternative approaches.

• Provide informed consent: Building up somewhat from the previous two
  points, subjects should be informed of and agree to the risks and benefits of
  participation before they join the study or experiment. This is also an
  opportunity for a participant to opt out of participation. However, there are
  some cases where deception is necessary. This might be to avoid biasing the
  participant's behavior by seeding their expectations, or if there is a dummy
  task surrounding the actual test to be performed. In cases like this, it's
  important to include a debriefing after the subject's participation so that
  they don't come away from the task feeling mislead.

• Handle sensitive data appropriately: If you're dealing with identifiable
  information in your study, make sure that you take appropriate steps to protect
  their anonymity from others. Sensitive information includes things like names,
  addresses, pictures, timestamps, and other links from personal identifiers to
  account information and history. Collected data should be anonymized as much as
  possible; surveys and census results are often also aggregated to avoid tracing
  outcomes back to any one person.

In the formal sciences, an experiment proposal must go through a review board
before it can be run, to ensure that ethical principles have been followed.
It's likely that you won't have a review board to submit your designs to prior
to running an experiment. You'll need to evaluate these principles for yourself
or with your colleagues to check for potential ethical issues before going
forward with a study design.

One particular point worth further discussion is that of informed consent for
web-based experiments. It's often the case that when an experiment is run,
users who are included in an experiment often don't know that they're
participating in an experiment. If a manipulation carries no risk and is so
minor as to be hidden away from the user (e.g. a change in recommendation
engine), perhaps there is no need for informed consent. And when it comes to
bias, it's known that peoples' behaviors can change when they know they are
under observation. In practice, informed consent is often not considered when
performing a web experiment.

However, informed consent is still an important ethical principle, so there is
continuing debate on how to best obtain consent for users of a website. One
option could be to allow users to opt out of experiment participation, with the
default user agreement implying consent to participation in unobtrusive
experiments. An opposing option would only run experiments on users who opt-in
to participation, asking the user to set their preference on their initial
visit or registration. The opt-in approach is more in line with the core idea
of informed consent, but also risks fewer users available for testing changes.

Examples in Experimental Ethics

Here are three studies in the social and medical sciences that are often
brought up as examples of violations of experimental ethics and progenitors of
the movement to establish ethics guidelines and boards:

• Tuskegee Syphilis Study:
  This study was started in 1932, where hundreds of African-American men were
  tracked over the course of up to forty years to study the natural progression
  of syphilis. The subjects were denied treatment, even after the development of
  effective syphilis treatments like penicillin, and there were active steps taken
  to hide the truth of their conditions and treatments to the subjects.
• Milgram obedience study:
  During the mid 1960s, psychologist Stanley Milgram tested to what degree people
  follow authority figures. Participants were asked to administer gradually
  increasing shocks to an acting confederate participant at the behest of a lead
  experimenter based on mistakes made on a dummy memory test used as a cover
  story. While no shocks were actually administered, the study did bing forth
  questions on what constitutes adequate debriefing and what level of information
  and informed consent needs to be provided to a participant in a study that
  includes necessary deceptive elements.
• Stanford Prison Experiment:
  This 1971 study conducted by psychologist Philip Zimbardo was built to test the
  dynamics and effects of power differences, using a prison scenario. Study
  volunteers were divided into prisoner and guard groups and their dynamics
  observed; the study had to be ended after less than a week due to the
  increasingly harsh conditions the 'guards' had settled into treating the
  'prisoners'. It has since served as a major ground for ethical criticisms,
  violating now-established guidelines around risks to participants and clarity
  of purpose. There are also methodological criticisms, as experimenter biases
  may have shaped the behavior of the 'guards' group in their interactions with
  'prisoners'.

And here's a Techcrunch article
discussing the ethics of the 2014-published Facebook study on the impact of
changing the affect of posts seen on users' feeds to their own posting habits.

Additional Resources

For further reading, here are a few links to documents both historical and
current on how to conduct experiments involving humans:

• Nuremberg Code -
  Ten principles for human subjects research stemming from the Nuremberg Trials
  published in 1949.
• The Belmont Report -
  1979 report laying out general principles for research involving human subjects
• APA Ethics Code - Guidelines published by
  the American Psychological Association for general psychological practices (not
  just research)
• BPS Code for Human Research Ethics -
  Code for human-subjects research published by the British Psychological Society